The "Patch" Mentality

Oracle's chief security officer made a recent speech, slamming the software development industry saying, "most software people are not trained to think in terms of safety, security and reliability." She complained about the "patch, patch, patch" culture, using the analogy of bridge construction:

"What if civil engineers built bridges the way developers write code?" she asked. "What would happen is that you would get the blue bridge of death appearing on your highway in the morning."

Well, I can only relate my own experiences.

One commenter on the story retorts, "Bridge makers don't have to put up with ever changing requirements." I am currently lead engineer on a fairly large project. Requirements were supposed to have been completed last December. We were still doing requirements analysis and modification well into April, a four month delay. Was the rest of the project delayed an equal amount? Of course not. Development was given an extra week. We've asked for another week, but I don't yet know if we will get it. So what we should have been doing in March, April, and May gets squeezed into part of May. Do you think that might cause some quality issues that will require subsequent patching?

I would take the requirements complaint a bit further. Not only is the business tardy in finalizing requirements, the requirements often don't make a lot of sense. A significant part of my current project is developing functionality that is all but duplicated in another area of the business. But we have to build it because our business clients think in terms of the process within our application rather than the duplicated process elsewhere.

Furthermore, we have other crazy requirements that are simply a reflection of the inefficient organization of the client area. They have different groups of people doing essentially the same thing. Therefore we have requirements to figure out who will get some report. It's the same report providing the same information, but we have figure out which of these groups gets the report. That adds cost, effort, and complexity to our project for so good reason other than to support the inefficient business organization. Complexity, in turn, leads to bugs and patches.

So, if my employer was serious about cutting costs and shortening projects, management would take a long, hard look at how the business side is structured and how requirements for projects are developed.

In fairness, IS does share in the blame for cost and complexity. Solutions are often overengineered. A few years ago I was lead engineer on a project to build a compensation system for some investment products. We went to the so-called experts, whose job it is to carefully analyze the application's needs and then to conclude that the application should our version of a J2EE application connecting to a UDB database. For that particular project, that would have meant multiple specialist teams (data specialists, source code management specialists, architecture specialists, user interface specialists, etc.) plus a 3-4 person development team working off a design I came up with. That approach would have taken at least 6 months, which would have been seriously pushing it, resulting in lots of bugs and subsequent patches. Total cost: at least $500k.

Instead, I built a Microsoft Access database, with me serving as all those specialists plus the 3-4 person development team (well, I did have two others writing the reports, but I was the development team for most of the project). The project was completed comfortably in about 4 months at a cost of about $100k. The system I delivered was so reliable and cheap that the business area subsequently added other products to it, and it's still around while the more officially sanctioned systems have been heavily worked on and, in some cases, rewritten. (This was supposed to be a short term solution until it could be rolled into an existing platform. That target platform has been subsequently replaced by a many million dollar project, and the replacement will again be replaced by another multi-million dollar effort. Meanwhile, my Access system just keeps on going and going.)

I'll be the first to tell you it is far from an elegant or pretty system. But there are times when an Access database or an Excel spreadsheet are all you need, and the IS sanctioned solutions are vastly more expensive and complex than needed. But few self-respecting IS professionals would ever want to build such systems. Few would even want to admit knowing Visual Basic.

There is a place for an emphasis on elegance and engineering. Absolutely. But sometimes that's overkill. Damiem Katz wrote about crappy programmers. Quite a few of those are situations where the programmer loses focus on what he or she needs to deliver and good design ideas become rigid rules that can never be violated, even if violating them makes a good deal of sense. (And some of the things on his list are things I battle frequently at work. "Enthusiastic UML modeling is typically done by those who aren't strong coders, but consider themselves software architects anyway." Can I please email that to a certain distribution list?) That was a recurring theme when I interviewed for my current position. I was asked several times what I would change, and my answer was always that I would change the mentality of "one size fits all" and get the engineers back to focusing on needs rather than design principles. If those principles don't apply to a situation, don't apply them.